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This  paper  is  based  on  an  article  which  first  appeared  in  ITEA 
Journal  of  Test  and  Evaluation,  Vol.  17,  Number  3,  Sept/Oct  96 

International  cooperation  in  test  and  evaluation  (T &E)  has  been  taking  place  for  the  past 
20  years  under  the  leadership  of  the  Office  of  the  Under  Secretary  of  Defense 
fAcauisition  and  Technolosv)  with  each  of  the  military  services  pursuing  aspects  of  the 
program.  Current  downsizing  trends  and  advances  in  weapons  technologies  present  the 
T&E  community  with  challenges  that  can  best  be  met  as  a  world  community  with 
programs  that  permit  the  sharing  of  assets — skills,  capabilities,  technologies,  and  ideas. 


T&E  has  demonstrated  its  value  over  the  years  as  an  area  for  international  cooperation. 
The  downsizing  taking  place  throughout  the  world's  military  establishments  has 
intensified  the  focus  in  this  area.  T&E,  with  its  large  industrial  structure  of  ranges  and 
facilities  and  high-technology  test  capabilities,  provides  great  potential  for  achieving 
economies  through  cooperation.  Indeed,  several  countries  are  considering  the  closure  of 
test  facilities  on  the  basis  of  capabilities  available  in  other  countries.  The  United  States  is 
also  exploiting  the  availability  of  allied  capabilities  to  preclude  costly  investments  in  new 
facilities.  The  concept  of  an  international  "reliance"  on  T&E  appears  to  be  emerging.  It  is 
prudent  to  seize  on  this  trend  and  promote  its  implementation. 

Several  instruments  are  already  in  place  that  provide  a  basis  for  pursuing  cooperative 
efforts  in  T&E.  A  Memorandum  of  Understanding  (MOU)  on  Mutual  Acceptance  of  Test 
and  Evaluation  among  the  four  powers  of  France,  Germany,  the  United  Kingdom,  and  the 
United  States,  has  been  in  place  since  1983  and  has  been  used  extensively  for 
development  of  mutual  test  procedures.  Similar  bilateral  MOU's  are  also  in  place  with 
Spain  and  Israel.  Additionally,  there  are  several  Data  Exchange  Agreements  (DEA's)  on 
proving  ground  techniques  with  the  countries  of  France,  Germany,  Sweden,  The 
Netherlands,  Israel,  South  Korea,  and  Australia.  Several  additional  MOU's  and  DEA's 
with  other  coimtries  are  currently  being  prepared  to  further  expand  the  base  of 
cooperative  test  and  evaluation  efforts. 
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Figure  1.  International  Test  Operations  Procedures 
Management  Structure 


There  are  many  opportunities  for  international  cooperation  in  T&E.  The  following 
provide  the  greatest  potential  for  economic  and  technical  benefit  to  the  international  T&E 
community. 

Test  Procedure  Standardization 

Test  procedure  standardization  is  a  proven  "enabler."  The  concept  has  been  in  existence 
for  more  than  10  years  under  the  International  Test  Operations  Procedures  (ITOP's) 
program  managed  by  the  countries  of  France,  Germany,  the  United  Kingdom,  and  the 
United  States.  This  program  has  produced  78  ratified  procedures,  with  an  estimated 
additional  50  in  various  stages  of  preparation.  Cooperatively  developed  by  world  subject 
matter  experts,  ITOP's  are  international  agreements  that  document  state-of-the-art  T&E 
procedures  for  testing  military  materiel.  The  use  of  ITOP's  provides  confidence  that  tests 
are  conducted  with  quality  procedures  and  produce  test  data  that  are  acceptable  by 
another  country  for  evaluation,  thus  avoiding  the  need  for  retesting  and  resulting  in  cost 
and  time  savings. 

Modeling  and  Simulation 

As  the  world  moves  increasingly  toward  the  cooperative  acquisition  of  military  and 
commercial  off-the-shelf  equipment,  technology  sharing,  and  coalition  operations, 
standardized  test  procedures  become  more  relevant  and  beneficial.  Rapid  advances  in 
weapons  and  testing  technology  make  them  even  more  crucial.  For  example,  modeling 


and  simulation  (M&S)  is  being  used  increasingly  in  both  materiel  development  and 
testing  with  new  applieations  being  explored  at  a  phenomenal  pace. 

Test  procedures  used  in  connection  with  testing  simulated  systems  and  subsystems  with 
equally  advanced  simulation  testing  tools  provide  a  great  degree  of  eomplexity  that  the 
tester  must  master.  The  expanding  use  of  modeling  and  simulation  in  the  development 
and  testing  of  complex  weapon  systems  allows  the  developer  to  explore  a  wide  variety  of 
alternative  designs  in  the  early  trade-off  of  key  technical  and  operational  features  at  the 
component  and  subsystem  level  and  at  the  system  level.  This  provides  the  developer  the 
means  of  quantifying  the  technological  risk  associated  with  a  particular  design,  and 
through  the  application  of  a  family  of  M&S  tools,  demonstrates  the  effectiveness  of  the 
particular  design  well  in  advance  of  actual  field  testing. 

Of  course,  the  M&S  tools  used  for  this  purpose  must  undergo  a  rigorous  certification 
process  better  known  as  verification  and  validation  (V&V).  The  V&V  process  demands 
the  M&S  results  be  corroborated  either  by  real  field  test  results  or  by  combat  itself  These 
certified  M&S  tools  must  be  able  to  predict  field  test  results  and  be  confirmed  by  those 
results.  Test  methods  and  procedures  play  a  crucial  role  in  this  traceability.  The 
international  implications  of  these  V&V  methods  and  procedures  will  be  particularly 
important  as  more  countries  apply  M&S  increasingly  to  shorten  their  development  and 
testing  efforts.  Consider  for  example,  a  system  developed  by  one  country  that  is  to  be 
purchased  by  another  country.  If  the  evaluation  of  that  system  was  based  on  data  obtained 
predominantly  from  simulation  trials,  a  costly  full-scale  retesting  program  may  be 
required  unless  the  purchasing  country  has  confidence  in  the  simulation  results. 

Technology  Management  Areas 

The  ITOP  program  encompasses  eight  technology  management  areas; 

(1)  Vehicles 

(2)  Weapons/ Ammunition 

(3)  Communications/Electronics 

(4)  Nuclear/Biological/Chemical  Protection 

(5)  Missiles/Rockets 

(6)  Aviation 

(7)  Modeling/Simulation 

(8)  Marine/Naval 

Each  area  is  managed  by  one  or  more  working  groups  depending  on  the  number  of 
subareas  it  includes  (as  shown  in  figure  1).  Management  areas  7  &  8  were  added  last 
year.  The  M&S  management  area  will  be  unique  since  it  will  not  directly  produce  M&S- 
based  ITOP's.  Instead  it  will  focus  on  M&S  fundamentals  such  as  the  architecture  and 
standards  necessary  for  integration  into  the  other  series  of  ITOP's. 

Program  Expansion 

The  success  of  the  ITOP  program  and  its  potential  for  expansion  is  being  exploited  to 
include  other  military  Services,  other  countries,  and  new  technology  areas. 

•  Other  Services. 

Initiated  predominantly  for  land  systems,  each  of  the  Four  Power  countries  has  expanded 
its  focus  to  include  air  and  sea  systems  ITOP  development. 


•  Other  countries. 

Several  additional  countries  have  expressed  interest  in  participating  in  the  development  of 
ITOP's.  Initially,  these  activities  will  be  pursued  as  bilateral  efforts  with  the  United 
States. 

•  New  technology  areas. 

M&S  and  Marine/Naval  were  added  last  year  as  a  new  technology  management  areas. 
Other  technology  areas  will  be  added  as  needed. 

Country  interactions  stemming  from  the  ITOP  program  have  spun  off  other  cooperative 
efforts  and  benefits.  For  example,  reciprocal  visits  between  countries  to  observe  testing 
and  assess  test  capabilities  firsthand  and  the  dialogue  with  international  colleagues  are 
important  aspects  of  international  T&E  since  they  instill  confidence  in  ITOP  results.  As 
testers  become  aware  of  test  and  simulation  capabilities  in  other  countries  and  the  testing 
challenges  facing  other  countries,  there  naturally  emerges  the  area  of  accessing  those 
capabilities  instead  of  investing  limited  resources  on  acquiring  duplicative  capabilities  at 
home.  There  have  also  been  a  number  of  occasions  when  one  country  has  become  aware 
of  a  new  test  technology  in  another  country  and  applied  that  technology  at  home  to  great 
benefit.  Not  only  has  this  saved  the  cost  of  development,  but  the  new  technology  may  not 
have  even  been  considered  had  it  not  become  known  through  ITOP  interaction.  Thus 
ITOP's,  while  focused  on  the  development  of  standard  test  procedures,  also  provide  a 
basis  for  other  international  cooperative  endeavors. 

Cooperative  Use  of  Test  and  Simulation  (T&S)  Facilities 

The  cooperative  use  of  T&S  facilities  provides  an  opportunity-rich  environment  for 
achieving  economies  and  efficiencies  through  the  use  of  allied  facilities  in  lieu  of 
investing  in  costly  new  facilities.  There  are  many  T&S  facilities  throughout  the  world 
that  are  one-of-a-kind  to  fit  a  particular  need  and  have  not  been  duplicated  because  of  the 
high  cost  involved  in  their  development  and  construction.  In  this  era  of  declining  defense 
budgets,  there  is  neither  investment  capital  nor  perhaps  the  volume  of  work  to  justify 
each  nation  possessing  all  of  the  capabilities  it  needs.  A  wiser  course  suggests  one 
capitalize  on  those  capabilities  already  in  existence  through  shared  use. 

While  this  idea  may  be  obvious  and  attractive,  putting  it  into  practice  will  require  some 
effort.  First,  international  agreements  will  have  to  be  reached  in  such  matters  as  pricing 
policy,  priority  of  use,  third-party  liability,  protection  of  intellectual  and  proprietary 
rights,  etc.  Second,  appropriate  information  on  test  capabilities  will  have  to  be  made 
available.  And  third,  a  process  will  have  to  be  put  in  place  to  implement  the  concept. 

Reciprocal  Agreements 

The  idea  of  using  another  country's  test  facilities  is  not  new.  Many  foreign  governments 
have  tested  their  systems  in  the  United  States,  and  the  United  States  has  tested  many  of  its 
systems  in  other  countries.  However,  these  instances  have  been  handled  on  a  case-by-case 
basis,  usually  under  "Foreign  Military  Sales"  arrangements.  Formal  reciprocal-use 
agreements  that  provide  equitable  price  structures  and  address  the  issues  of  testing 
priority,  logistics,  intellectual  property  rights,  and  other  legal  matters  need  to  be 
developed  to  facilitate  the  process.  Many  countries  are  expressing  this  line  of  thought. 
Such  an  agreement  already  exists  under  a  Diplomatic  Exchange  of  Notes  with  Canada 
called  the  Canadian-United  States  Test  and  Evaluation  Program.  It  has  been  highly 
successful  for  more  than  10  years! 


Directories  of  Test  Capabilities 

Recognizing  that  testers  must  be  familiar  with  test  capabilities  in  other  countries  before 
those  capabilities  can  be  sought  and  used,  the  U.S.  Army  Test  and  Evaluation  Command 
(TECOM)  in  1990,  produced  an  International  Test  Facilities  and  Ranges  Summary  of 
Capabilities  catalog.  Originally  covering  test  capabilities  of  five  countries,  a  recently 
completed  revision  now  under  a  U.S.  Department  of  Defense  logo,  expands  coverage  to 
eight  countries:  Canada,  France,  Germany,  the  United  Kingdom,  United  States,  Israel, 
Australia,  and  Sweden. 

The  European  test  community  is  also  moving  in  the  direction  of  shared  test  facilities.  The 
Western  European  Armaments  Group  (WEAG)  has  just  published  a  "Directory  of  Test 
Centres."  The  group  is  composed  of  thirteen  countries:  Belgium,  Denmark,  France, 
Germany,  Greece,  Italy,  Luxembourg,  Netherlands,  Norway,  Portugal,  Spain,  Turkey, 
and  the  United  Kingdom.  Collectively,  these  documents  describe  test  capabilities  at  over 
200  test  facilities  and  centers  in  thirteen  countries.  The  catalogs  and  directories  will 
undoubtedly  foster  and  facilitate  reciprocal  use  of  test  facilities.  Ultimately,  these 
documents  will  be  made  available  via  the  World  Wide  Web,  which  will  make  them  even 
more  accessible. 

While  there  has  been  considerable  discussion  between  countries  on  the  subject  of 
cooperative  use  of  test  facilities,  efforts  to  date  have  been  ad  hoc  since  there  is  no  formal 
organized  group  in  many  of  the  countries  to  promote  and  oversee  such  a  process.  This 
may  ultimately  be  required. 

Test  Technology  (TT)  Transfer 

TT  transfer  is  a  unique  area  that  has  been  little  exploited.  The  development  and 
acquisition  of  test  technologies  (i.e.,  instrumentation  and/or  procedures)  are  complex, 
costly,  and  time  consuming.  Each  country  has  many  unique  and  specialized  test 
technologies  under  development.  If  already  developed  technologies  are  shared,  they  can 
greatly  reduce  future  cost  by  eliminating  redundant  development.  There  are  many 
examples  which  clearly  illustrate  these  benefits  and  potentials,  but  most  of  these 
technology  transfers  occurred  by  happenstance  -  the  result  of  one  country's  accidentally 
becoming  aware  of  a  new  test  technology  in  another  country  and  adapting  or  purchasing 
it.  Technology  transfers  can  be  accomplished  with  varied  approaches.  A  few  examples 
are  described  below. 

Direct  Technology  Exchange 

The  United  States,  France,  and  Sweden  are  exchanging  information  on  range 
technologies.  The  United  States  is  evaluating  French  and  Swedish  projectile  catch  box 
designs  for  application  to  depleted  uranium  test  firings  in  the  United  States.  These 
designs,  which  use  self-cleaning  technology,  have  the  potential  of  saving  millions  of 
dollars  annually  in  clean  up  of  depleted  uranium  residuals.  Reciprocally,  France  and 
Sweden  are  interested  in  U.S.  developments  in  acoustic  scoring  technology  and  the 
Hardened  Subminiature  Telemetry  and  Sensor  System. 

Collaborative  Technology  Effort 

The  United  States  and  Germany  collaborated  on  the  development  of  new  ballistic  shock 
protection  specification  requirements  for  armored  vehicles.  These  specifications. 


developed  through  test  technology  research  and  development,  provide  new  insights  into 
shock  phenomena  which  negate  previous,  erroneous  specifications.  This  collaborative 
effort  saved  the  United  States  an  estimated  $1  million.  Significant  savings  have  also  been 
realized  in  development  of  the  Abrams  Tank  and  the  Crusader  advanced  field  artillery 
system  through  more  accurate  design  by  using  the  new  specifications. 

Reiterative  Technology  Sharing 

The  Projectile  Follower  instrument  which  is  capable  of  providing  visual  in-flight 
performance  data  of  high-speed,  low-trajectory  projectiles,  is  another  example  of  shared 
test  technology.  Originally  designed  in  the  United  Kingdom,  the  Projectile  Follower 
provided  the  genesis  for  a  modified  U.S.  design.  The  U.S.  design  in  turn  provided  the 
springboard  for  a  further  design  improvement  in  the  United  Kingdom  system.  This 
"rotating"  of  technology  between  the  two  countries  resulted  in  continuing  technology 
maturation  benefiting  both  countries. 

There  has  been  no  formal  effort  directed  at  exploiting  the  potential  of  international  test 
technology  sharing.  This  is  clearly  a  promising  area  that  should  be  pursued. 

The  Future 

International  cooperation  is  crucial  as  governments  downsize.  We  have  seen  the  necessity 
and  value  of  this  as  we  have  brought  our  testers  and  developers  together  to  work  more 
closely  with  each  other,  as  we  are  now  bringing  the  Services  together  to  intensify 
interservice  cooperation,  and  as  we  strive  to  form  partnerships  with  our  industry. 

We  must  also  apply  this  philosophy  internationally.  Yes,  it  may  be  a  little  more  complex 
to  accomplish  -  agreements  are  needed  to  address  legalities,  logistics,  and  security  _  and 
there  are  considerable  distances  over  which  to  travel  and  communicate.  But  the  benefits 
make  the  effort  worthwhile. 
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